Structure of Code

To sign people on, you need their boat number, and their boat name, from there you can work out their name and sign them on completely.

Boat Or No Boat?

The code starts by determining whether there is a boat in the picture, using a multi-layered machine learning sigmoid model. This model is trained on about 1000 pieces of training images. If there isn't, it will not waste its time trying to decipher boat numbers from water.

Boat Outliner

Then, it will give me coordinates to an outline of the boat (to crop it from the rest of the image) using a linear regression model, with help from VGG16. This is to make it easier for the later models to make correct decisions. This model is trained on about a couple thousand training images. This is done as illustrated here:

Boat Recogniser

From that cropped image, another softmax model would work out the type of boat it is, which is one component to signing people on. The current boat types identifiable by the model are: 29er, Laser, Solo, Aero, Tera and Feva. In later versions, I would make more categories as there a alot more boats than that, but those are the popular boats.

Boat Num Outliner Model

Also from that cropped image, I would draw an outline, with another VGG16 model, onto the boat number on the sail, as shown in this picture. This would be again, cropped from the boat, and passed to the next model.

Boat Num Divider

This will take the cropped boat number from the cropped boat, and will use opencv contours, rather than a ML model, to further split all numbers into individual numbers, as shown in the image to the right, which could then passed to the next number reader model, which actually reads each number into the system.

Number Reader

This model would get all cropped numbers from the cropped boat number, and read each one. There is a category for "None" as it is possible for the contours to pick up random black lines from the cropped boat number, as shown by the furthest right image. Then, the number (869) is passed to the step in the process.

Sign On to website

Finally, the numbers are converted into a dictionary, meaning that they do not care for order (as it is impossible to have it in order), and are compared to all numbers (also as dictionaries) in the online graphql database, which I accessed through the use of the requests python library. There is a points system which I put in place to give rarer boats more chance to be signed on with numbers that are a bit off, as there is less chance of signing the wrong person on. For example, the Laser, which is a common boat, would need a more harsh points system as there are a lot of people with similar sail numbers, meaning I wouldn't want to sign someone who is not racing, but has similar sail numbers, on to the race. This is only needed as I need to give the models a chance to actually read any boats on to the website. The boat should hopefully be signed on from there.

Overview

First, a boat is decided to be in the picture, then it is outlined, then it's type is decided, and boat number outined, both from the cropped image. Next, the cropped boat number is then divided into its numbers, and then those numbers are read. The full number is judged as to whether it is accurate enough to any boats(in the database), and is then signed on.

Notes

I hope to have tried my model on some real-life data by the end of march, and improve it from there to actually be functional, but apart from that the project has been incredibly fun for me to work on, as it is something I love, Sailing, and learning about Machine Learning has been fascinating!